{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. 观察Otto商品的特征进行PCA各维的方差，可以得到什么结论？（20分） \n",
    "2. 对Otto商品tfidf特征，进行PCA降维，给出各维方差的分布图。（30分） \n",
    "3. 采用train_test_split，从将数据集中随机抽取10000条记录（原始数据集太大，剩余数据抛弃，此部分SVM作业已经完成）。对这部分数据进行PCA降维，保留85%的能量。（20分） \n",
    "4. 对3中得到的数据（对降维后的数据），训练RBF核SVM，并对超参数（C和gamma）进行超参数调优。结果和用原始数据的情况比较（SVM部分作业结果）。（30分） \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1.所用:plt.bar(range(len(pca.explained_variance_ratio_)), pca.explained_variance_ratio_)\n",
    "绘制出来的直方图，横轴为特征维度的索引，纵轴为每一维度的方差占总能量的比例。越重要的特征所占比例越高。\n",
    "由图，从左至右特征方差从大至小排列，知前面包含的信息量多。\n",
    "PCA是希望投影后的投影值尽量分散，而这种分散程度，可以由方差来表示。\n",
    "我们得到了降维问题的优化目标：将一组N维向量降为 K 维（ K 大于0，小于 N ），其目标是选择K个单位（模为1）正交基，使得原始数据变换到这组基上后，各字段两两间协方差为0，而字段的方差则尽可能大（在正交的约束下，取最大的K个方差）。\n",
    "\n",
    "2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.decomposition import PCA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>feat_1</th>\n",
       "      <th>feat_2</th>\n",
       "      <th>feat_3</th>\n",
       "      <th>feat_4</th>\n",
       "      <th>feat_5</th>\n",
       "      <th>feat_6</th>\n",
       "      <th>feat_7</th>\n",
       "      <th>feat_8</th>\n",
       "      <th>feat_9</th>\n",
       "      <th>...</th>\n",
       "      <th>feat_85</th>\n",
       "      <th>feat_86</th>\n",
       "      <th>feat_87</th>\n",
       "      <th>feat_88</th>\n",
       "      <th>feat_89</th>\n",
       "      <th>feat_90</th>\n",
       "      <th>feat_91</th>\n",
       "      <th>feat_92</th>\n",
       "      <th>feat_93</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Class_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Class_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Class_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Class_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>Class_1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 95 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  feat_1  feat_2  feat_3  feat_4  feat_5  feat_6  feat_7  feat_8  feat_9  \\\n",
       "0   1       1       0       0       0       0       0       0       0       0   \n",
       "1   2       0       0       0       0       0       0       0       1       0   \n",
       "2   3       0       0       0       0       0       0       0       1       0   \n",
       "3   4       1       0       0       1       6       1       5       0       0   \n",
       "4   5       0       0       0       0       0       0       0       0       0   \n",
       "\n",
       "    ...     feat_85  feat_86  feat_87  feat_88  feat_89  feat_90  feat_91  \\\n",
       "0   ...           1        0        0        0        0        0        0   \n",
       "1   ...           0        0        0        0        0        0        0   \n",
       "2   ...           0        0        0        0        0        0        0   \n",
       "3   ...           0        1        2        0        0        0        0   \n",
       "4   ...           1        0        0        0        0        1        0   \n",
       "\n",
       "   feat_92  feat_93   target  \n",
       "0        0        0  Class_1  \n",
       "1        0        0  Class_1  \n",
       "2        0        0  Class_1  \n",
       "3        0        0  Class_1  \n",
       "4        0        0  Class_1  \n",
       "\n",
       "[5 rows x 95 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_path= open(r'C:\\Users\\dell\\Downloads\\201913171344570_36639\\3代码类素材\\Otto_PCA_TNSE\\Otto_PCA_TNSE\\data\\Otto_train.csv')\n",
    "train=pd.read_csv(train_path)\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feat_1</th>\n",
       "      <th>feat_2</th>\n",
       "      <th>feat_3</th>\n",
       "      <th>feat_4</th>\n",
       "      <th>feat_5</th>\n",
       "      <th>feat_6</th>\n",
       "      <th>feat_7</th>\n",
       "      <th>feat_8</th>\n",
       "      <th>feat_9</th>\n",
       "      <th>feat_10</th>\n",
       "      <th>...</th>\n",
       "      <th>feat_84</th>\n",
       "      <th>feat_85</th>\n",
       "      <th>feat_86</th>\n",
       "      <th>feat_87</th>\n",
       "      <th>feat_88</th>\n",
       "      <th>feat_89</th>\n",
       "      <th>feat_90</th>\n",
       "      <th>feat_91</th>\n",
       "      <th>feat_92</th>\n",
       "      <th>feat_93</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.080436</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.074055</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.227278</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.196169</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.011846</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.010756</td>\n",
       "      <td>0.103045</td>\n",
       "      <td>0.021145</td>\n",
       "      <td>0.073024</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.013132</td>\n",
       "      <td>...</td>\n",
       "      <td>0.446712</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.008051</td>\n",
       "      <td>0.022456</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.121616</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.142568</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 93 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     feat_1  feat_2  feat_3    feat_4    feat_5    feat_6    feat_7    feat_8  \\\n",
       "0  0.080436     0.0     0.0  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
       "1  0.000000     0.0     0.0  0.000000  0.000000  0.000000  0.000000  0.227278   \n",
       "2  0.000000     0.0     0.0  0.000000  0.000000  0.000000  0.000000  0.196169   \n",
       "3  0.011846     0.0     0.0  0.010756  0.103045  0.021145  0.073024  0.000000   \n",
       "4  0.000000     0.0     0.0  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
       "\n",
       "   feat_9   feat_10   ...      feat_84   feat_85   feat_86   feat_87  feat_88  \\\n",
       "0     0.0  0.000000   ...     0.000000  0.074055  0.000000  0.000000      0.0   \n",
       "1     0.0  0.000000   ...     0.000000  0.000000  0.000000  0.000000      0.0   \n",
       "2     0.0  0.000000   ...     0.000000  0.000000  0.000000  0.000000      0.0   \n",
       "3     0.0  0.013132   ...     0.446712  0.000000  0.008051  0.022456      0.0   \n",
       "4     0.0  0.000000   ...     0.000000  0.121616  0.000000  0.000000      0.0   \n",
       "\n",
       "   feat_89   feat_90  feat_91  feat_92  feat_93  \n",
       "0      0.0  0.000000      0.0      0.0      0.0  \n",
       "1      0.0  0.000000      0.0      0.0      0.0  \n",
       "2      0.0  0.000000      0.0      0.0      0.0  \n",
       "3      0.0  0.000000      0.0      0.0      0.0  \n",
       "4      0.0  0.142568      0.0      0.0      0.0  \n",
       "\n",
       "[5 rows x 93 columns]"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.feature_extraction.text import TfidfTransformer\n",
    "tfidf = TfidfTransformer()\n",
    "X_train = train.drop([\"id\", \"target\"], axis=1)\n",
    "train_id = train['id']\n",
    "#输出稀疏矩阵\n",
    "X_train_tfidf = tfidf.fit_transform(X_train).toarray()\n",
    "columns_name=X_train.columns\n",
    "X_train_tfidf=pd.DataFrame(columns=columns_name,data=X_train_tfidf)\n",
    "X_train_tfidf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#PCA降维\n",
    "pca = PCA(n_components = 0.85)\n",
    "pca.fit(X_train_tfidf)\n",
    "\n",
    "X_train_pca = pca.transform(X_train_tfidf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAD/ZJREFUeJzt3X+s3Xddx/Hny5ZtCHGDrRpcW1uy\naigBUUshEZVsOjpBSuKWdPijJjPVhBmMEOxMGFAxYcZQ/rAmNm66DHVbhj9upLosG/4IIaMdE0aZ\nDZeJ7LqFdbYMJ9lGx9s/znfh9Ox093tvT+/tPZ/nI2n6/X6+n+85n/NJ+/p+zuec7+ekqpAkteF7\nlrsBkqSlY+hLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGrJ6uRsw6qKLLqoNGzYs\ndzMkaUW57777Hq+qNfPVO+tCf8OGDRw6dGi5myFJK0qS/+pTz+kdSWqIoS9JDTH0Jakhhr4kNcTQ\nl6SGGPqS1BBDX5IaYuhLUkMMfUlqyFl3R+7p2rD7k88r++pH3roMLZGks48jfUlqiKEvSQ0x9CWp\nIYa+JDXE0Jekhhj6ktSQXqGfZFuSI0lmk+wec/ynk3wuyYkkV44c25nky92fnZNquCRp4eYN/SSr\ngH3AFcBm4Ookm0eqfQ34NeCvRs59OfAB4A3AVuADSV52+s2WJC1Gn5H+VmC2qh6qqmeAW4HtwxWq\n6qtV9QXgOyPnvgW4q6qOVdVx4C5g2wTaLUlahD6hfzHw8ND+XFfWR69zk+xKcijJoaNHj/Z8aEnS\nQvUJ/Ywpq56P3+vcqtpfVVuqasuaNfP+mLskaZH6hP4csG5ofy3wSM/HP51zJUkT1if0DwKbkmxM\ncg6wA5jp+fh3ApcneVn3Ae7lXZkkaRnMG/pVdQK4lkFYPwjcXlWHk+xJ8naAJK9PMgdcBfxpksPd\nuceA32dw4TgI7OnKJEnLoNfSylV1ADgwUnb90PZBBlM34869CbjpNNooSZoQ78iVpIYY+pLUEENf\nkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWp\nIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi\n6EtSQwx9SWqIoS9JDekV+km2JTmSZDbJ7jHHz01yW3f83iQbuvIXJbk5yQNJHkxy3WSbL0laiHlD\nP8kqYB9wBbAZuDrJ5pFq1wDHq+oSYC9wQ1d+FXBuVb0G+AngN567IEiSll6fkf5WYLaqHqqqZ4Bb\nge0jdbYDN3fbdwCXJQlQwEuSrAZeDDwDfHMiLZckLVif0L8YeHhof64rG1unqk4ATwAXMrgA/B/w\nKPA14I+q6thptlmStEh9Qj9jyqpnna3As8APAhuB9yR55fOeINmV5FCSQ0ePHu3RJEnSYvQJ/Tlg\n3dD+WuCRU9XppnLOB44B7wT+qaq+XVWPAZ8Gtow+QVXtr6otVbVlzZo1C38VkqRe+oT+QWBTko1J\nzgF2ADMjdWaAnd32lcA9VVUMpnQuzcBLgDcC/zGZpkuSFmre0O/m6K8F7gQeBG6vqsNJ9iR5e1ft\nRuDCJLPA7wDPfa1zH/BS4IsMLh5/XlVfmPBrkCT1tLpPpao6ABwYKbt+aPspBl/PHD3vyXHlkqTl\n4R25ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqI\noS9JDTH0Jakhhr4kNcTQl6SG9PoRlWmwYfcnx5Z/9SNvXeKWSNLycaQvSQ0x9CWpIYa+JDXE0Jek\nhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUkF6hn2RbkiNJZpPsHnP8\n3CS3dcfvTbJh6Nhrk3wmyeEkDyQ5b3LNlyQtxLyhn2QVsA+4AtgMXJ1k80i1a4DjVXUJsBe4oTt3\nNfBx4Der6tXAm4FvT6z1kqQF6TPS3wrMVtVDVfUMcCuwfaTOduDmbvsO4LIkAS4HvlBVnweoqv+p\nqmcn03RJ0kL1Cf2LgYeH9ue6srF1quoE8ARwIfDDQCW5M8nnkrzv9JssSVqsPr+clTFl1bPOauBN\nwOuBbwF3J7mvqu4+6eRkF7ALYP369T2aNFn+qpakVvQZ6c8B64b21wKPnKpON49/PnCsK/+Xqnq8\nqr4FHAB+fPQJqmp/VW2pqi1r1qxZ+KuQJPXSJ/QPApuSbExyDrADmBmpMwPs7LavBO6pqgLuBF6b\n5Hu7i8HPAF+aTNMlSQs17/ROVZ1Ici2DAF8F3FRVh5PsAQ5V1QxwI3BLklkGI/wd3bnHk3yUwYWj\ngANVNX4uRZJ0xvWZ06eqDjCYmhkuu35o+yngqlOc+3EGX9uUJC0z78iVpIYY+pLUEENfkhpi6EtS\nQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1JBeyzC0bNyyyy65LGmlcqQvSQ0x9CWpIYa+JDXE\n0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xLV3ToPr8khaaRzpS1JDDH1Jaoih\nL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhrSK/STbEtyJMlskt1jjp+b5Lbu+L1JNowcX5/kySTv\nnUyzJUmLMW/oJ1kF7AOuADYDVyfZPFLtGuB4VV0C7AVuGDm+F/jH02+uJOl09BnpbwVmq+qhqnoG\nuBXYPlJnO3Bzt30HcFmSACR5B/AQcHgyTZYkLVaf0L8YeHhof64rG1unqk4ATwAXJnkJ8LvAh17o\nCZLsSnIoyaGjR4/2bbskaYH6hH7GlFXPOh8C9lbVky/0BFW1v6q2VNWWNWvW9GiSJGkx+qyyOQes\nG9pfCzxyijpzSVYD5wPHgDcAVyb5Q+AC4DtJnqqqPz7tlkuSFqxP6B8ENiXZCPw3sAN450idGWAn\n8BngSuCeqirgp56rkOSDwJMGviQtn3lDv6pOJLkWuBNYBdxUVYeT7AEOVdUMcCNwS5JZBiP8HWey\n0ZKkxen1IypVdQA4MFJ2/dD2U8BV8zzGBxfRvhXLH1iRdDbyjlxJaoihL0kNMfQlqSGGviQ1xNCX\npIYY+pLUkF5f2dTkjPsqJ/h1TklLw5G+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQl\nqSHenHUW8cYtSWeaI31JaoihL0kNcXpnhfDnFyVNgiN9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS\n1BBDX5IaYuhLUkMMfUlqiHfkTgHv1pXUlyN9SWpIr9BPsi3JkSSzSXaPOX5uktu64/cm2dCV/1yS\n+5I80P196WSbL0laiHlDP8kqYB9wBbAZuDrJ5pFq1wDHq+oSYC9wQ1f+OPALVfUaYCdwy6QaLkla\nuD4j/a3AbFU9VFXPALcC20fqbAdu7rbvAC5Lkqq6v6oe6coPA+clOXcSDZckLVyfD3IvBh4e2p8D\n3nCqOlV1IskTwIUMRvrP+UXg/qp6evHN1UL4S1ySRvUJ/Ywpq4XUSfJqBlM+l499gmQXsAtg/fr1\nPZokSVqMPqE/B6wb2l8LPHKKOnNJVgPnA8cAkqwF/hb41ar6yrgnqKr9wH6ALVu2jF5QdAb4LkBq\nU585/YPApiQbk5wD7ABmRurMMPigFuBK4J6qqiQXAJ8ErquqT0+q0ZKkxZl3pN/N0V8L3AmsAm6q\nqsNJ9gCHqmoGuBG4JcksgxH+ju70a4FLgPcneX9XdnlVPTbpF6LJ8V2ANL163ZFbVQeAAyNl1w9t\nPwVcNea8DwMfPs026izi3b/SyuYduZLUEENfkhrigmuaGKd+pLOfI31JaoihL0kNMfQlqSHO6euM\n83v/0tnDkb4kNcSRvpaV7wKkpWXo66zlV0ClyTP0tSK90AXBi4V0aoa+muFUkmToS4AXBLXD0Jfm\n4VSSpomhL50hXhB0NjL0pSX2QlNJTjPpTDP0pRXCdw6aBENfmgKL+dzBdxVtMvQlPc9Cp6D8YHvl\nMPQlLYnFvONY7MVHp2boS5o6vhs5NUNfklj8t6pW2kXE0JekM+Rs/BDd9fQlqSGGviQ1xNCXpIYY\n+pLUEENfkhpi6EtSQwx9SWpIr9BPsi3JkSSzSXaPOX5uktu64/cm2TB07Lqu/EiSt0yu6ZKkhZo3\n9JOsAvYBVwCbgauTbB6pdg1wvKouAfYCN3TnbgZ2AK8GtgF/0j2eJGkZ9BnpbwVmq+qhqnoGuBXY\nPlJnO3Bzt30HcFmSdOW3VtXTVfWfwGz3eJKkZdAn9C8GHh7an+vKxtapqhPAE8CFPc+VJC2RVNUL\nV0iuAt5SVb/e7f8KsLWqfmuozuGuzly3/xUGI/o9wGeq6uNd+Y3Agar6xMhz7AJ2dbs/AhyZwGu7\nCHh8Ao8zLeyPk9kfJ7M/TrYS++OHqmrNfJX6LLg2B6wb2l8LPHKKOnNJVgPnA8d6nktV7Qf292hL\nb0kOVdWWST7mSmZ/nMz+OJn9cbJp7o8+0zsHgU1JNiY5h8EHszMjdWaAnd32lcA9NXgLMQPs6L7d\nsxHYBHx2Mk2XJC3UvCP9qjqR5FrgTmAVcFNVHU6yBzhUVTPAjcAtSWYZjPB3dOceTnI78CXgBPCu\nqnr2DL0WSdI85p3TX6mS7OqmjYT9Mcr+OJn9cbJp7o+pDX1J0vO5DIMkNWQqQ3++ZSOmXZKbkjyW\n5ItDZS9PcleSL3d/v2w527iUkqxL8qkkDyY5nOTdXXmTfZLkvCSfTfL5rj8+1JVv7JZR+XK3rMo5\ny93WpZJkVZL7k/xDtz+1fTF1od9z2Yhp9xcMlr0Ythu4u6o2AXd3+604Abynql4FvBF4V/dvotU+\neRq4tKp+FHgdsC3JGxksn7K364/jDJZXacW7gQeH9qe2L6Yu9Om3bMRUq6p/ZfAtqmHDS2XcDLxj\nSRu1jKrq0ar6XLf9vwz+c19Mo31SA092uy/q/hRwKYNlVKCh/kiyFngr8GfdfpjivpjG0Hfph/F+\noKoehUEIAt+/zO1ZFt0KsD8G3EvDfdJNZ/w78BhwF/AV4BvdMirQ1v+bjwHvA77T7V/IFPfFNIZ+\nxpT5FSWR5KXAJ4DfrqpvLnd7llNVPVtVr2Nwl/xW4FXjqi1tq5ZekrcBj1XVfcPFY6pOTV/0WYZh\npem19EODvp7kFVX1aJJXMBjhNSPJixgE/l9W1d90xU33CUBVfSPJPzP4rOOCJKu7EW4r/29+Enh7\nkp8HzgO+j8HIf2r7YhpH+n2WjWjR8FIZO4G/X8a2LKlujvZG4MGq+ujQoSb7JMmaJBd02y8GfpbB\n5xyfYrCMCjTSH1V1XVWtraoNDLLinqr6Jaa4L6by5qzuqv0xvrtsxB8sc5OWVJK/Bt7MYKXArwMf\nAP4OuB1YD3wNuKqqRj/snUpJ3gT8G/AA3523/T0G8/rN9UmS1zL4cHIVg4Hf7VW1J8krGXzx4eXA\n/cAvV9XTy9fSpZXkzcB7q+pt09wXUxn6kqTxpnF6R5J0Coa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JD\nDH1JaoihL0kN+X8PyN4h0IyEFQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2bc8b739320>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#绘制方差\n",
    "plt.bar(range(len(pca.explained_variance_ratio_)), pca.explained_variance_ratio_)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(61878, 93)"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train_tfidf.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10000, 93)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_split.py:2026: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "y_train = train['target']   \n",
    "X_train_part,_,_,_ =train_test_split(X_train_tfidf,y_train,train_size=10000, random_state=10)\n",
    "print(X_train_part.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "pca = PCA(n_components = 0.85)\n",
    "pca.fit(X_train_part)\n",
    "\n",
    "X_train_pca = pca.transform(X_train_part)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#保存PCA特征变换结果\n",
    "n_components = pca.n_components_\n",
    "feat_names_pca = []\n",
    "for i in range(n_components):\n",
    "    feat_names_pca.append(\"pca_\" + str(i))\n",
    "\n",
    "y = pd.Series(data = y_train, name = 'target')\n",
    "train_pca = pd.concat([train_id, pd.DataFrame(columns = feat_names_pca, data = X_train_pca), y], axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.svm import SVC\n",
    "def fit_grid_point_RBF(C, gamma, X_train, y_train, X_test, y_test):\n",
    "\n",
    "    SVC3 = SVC( C = C, kernel='rbf', gamma = gamma)\n",
    "    SVC3 = SVC3.fit(X_train, y_train)\n",
    "    accuracy = SVC3.score(X_test, y_test)\n",
    "    \n",
    "    print(\"C= {} and gamma = {}: accuracy= {} \" .format(C, gamma, accuracy))\n",
    "    return accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'\\n# 用核PCA方法来降维\\nfrom sklearn.decomposition import KernelPCA\\nkernel_pca = KernelPCA(kernel=\"rbf\", fit_inverse_transform=True, gamma=10)\\nX_kernel_pca = kernel_pca.fit_transform(X_train_pca)\\nprint(X_kernel_pca.shape)\\nvisual_2D_dataset(X_kernel_pca[:,:2],dataset_y,\\'Kernel PCA transformed dataset\\')\\n'"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'''\n",
    "# 用核PCA方法来降维\n",
    "from sklearn.decomposition import KernelPCA\n",
    "kernel_pca = KernelPCA(kernel=\"rbf\", fit_inverse_transform=True, gamma=10)\n",
    "X_kernel_pca = kernel_pca.fit_transform(X_train_pca)\n",
    "print(X_kernel_pca.shape)\n",
    "visual_2D_dataset(X_kernel_pca[:,:2],dataset_y,'Kernel PCA transformed dataset')\n",
    "'''"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#寻找最佳参数\n",
    "accuracy_s = np.matrix(np.zeros(shape=(5, 3)), float)\n",
    "gamma_s = np.logspace(-1, 1, 3)  \n",
    "oneC = 0.1\n",
    "\n",
    "for j, gamma in enumerate(gamma_s):\n",
    "    accuracy_s[0,j] = fit_grid_point_RBF(oneC, gamma, X_train_part,y_train, X_test,y_test)\n",
    "\n",
    "oneC = 1\n",
    "\n",
    "for j, gamma in enumerate(gamma_s):\n",
    "    accuracy_s[0,j] = fit_grid_point_RBF(oneC, gamma, X_train_part,y_train, X_test,y_test)\n",
    "\n",
    "oneC = 10\n",
    "\n",
    "for j, gamma in enumerate(gamma_s):\n",
    "    accuracy_s[0,j] = fit_grid_point_RBF(oneC, gamma, X_train_part,y_train, X_test,y_test)\n",
    "    \n",
    "oneC = 100\n",
    "\n",
    "for j, gamma in enumerate(gamma_s):\n",
    "    accuracy_s[0,j] = fit_grid_point_RBF(oneC, gamma, X_train_part,y_train, X_test,y_test)\n",
    "    \n",
    "    "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
